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Abstract — Orthogonal matching pursuit (OMP) is a greedy 
algorithm widely used for the recovery of sparse signals from 
compressed measurements. In this paper, we consider the perfect 
recovery of A-sparse signals using the OMP algorithm for two 
distinct scenarios, viz., when it performs A iterations and more 
than K iterations. In the first part of this paper, we show that if 
the sensing matrix satisfies the restricted isometry property (RIP) 
with \/K9k,i + 8k < 1 then the OMP algorithm can perfectly 
recover any A-sparse signal in K iterations. Our result bridges 
the mutual incoherence property (MIP) and RIP based recovery 
bounds and also embraces many of recently proposed recovery 
bounds of the OMP. In the second part of this paper, we show that 
if the sensing matrix satisfies the RIP with 5[s.93Jfj < 0.03248, 
then the OMP perfectly recovers any A'-sparse signal within 6A 
iterations. 

Index Terms — Sparse recovery, orthogonal matching pursuit 
(OMP), restricted isometric property (RIP), mutual incoherence 
property (MIP), compressive sensing (CS) 



I. Introduction 

Recently there has been a growing interest in recovering 
sparse signals from compressed measurements ifTI- lfTTl . The 
main goal of this task is to accurately estimate a high di- 
mensional A'-sparse vector x £ 1Z n (||x||o < K) from a 
small number of linear measurements y £ lZ m (m <C n). 
The relationship between the signal vector and measurements 
is 

y = *x (l) 

where «I> <E -fcmxn j s ft en called the sensing matrix. Since 
this system is underdetermined, infinitely many solutions exist 
and hence one cannot accurately recover the original signal 
vector x. However, due to the prior information of signal 
sparsity, the signal vector x can be accurately reconstructed 
via properly designed recovery algorithms. Among many 
recovery algorithms in the literature, orthogonal matching 
pursuit (OMP) algorithm has generated a great deal of interest 
in recent years for its competitive performance as well as 
practical benefits such as implementation simplicity and low 
computational complexity [5|, |10j, lfl2l . 

The OMP algorithm iteratively estimates the sparse signal x 
and its support (i.e., index set of nonzero elements). Suppose 
the If -sparse vector x is supported on T and let T k , x fc and 
r fe be the estimated support, the estimated sparse signal, and 
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the residual in the /c-th iteration, respectively. Then the OMP 
repeats the following operations until the residual equals zero 
or the iteration number reaches a predefined maximum value: 
• Among correlations between fa (z-th column of 3?) and 
the residual r fe_1 generated in the (k— l)-th iteration, find 
the largest element in magnitude and the corresponding 
index t k . That is, 



t 



arg max 

ie{i,-- ,n}\T fc " 



\{r*-\fa 



(2) 



Once t k is identified, add this index into the estimated 
support set 

T k = T fc_1 U {t k }. (3) 
Solve the least squares (LS) problem 

"y-*v|L. 



arg mm | 

supp(v)—T k 

Update the residual of the fc-th iteration as 
r fe = v - *x fe . 



(4) 



(5) 



Readers are referred to Ifl2l for more details. 

The OMP algorithm has long been considered as a heuristic 
algorithm hard to be analyzed. Recently, however, many efforts 
have been made to discover conditions of the OMP ensuring 
the perfect recovery of sparse signals. In the analysis, a 
condition so called the mutual incoherence property (MIP) has 
been employed [1|. The mutual coherence parameter of 
the sensing matrix €> is defined as 



M(*) 



max | {fa, fa) | 



where fa and fa are two distinct column vectors of €>. Tropp 
showed that if €> has unit £2 -norm columns and satisfies 
/i(4>) < 2A 3_ 1 , then the OMP algorithm can reconstruct K- 
sparse signals accurately in K iterations lfl2l . An alternate and 
recently popular framework is the restricted isometric property 
(RIP) [2|. A sensing matrix $ is said to satisfy the RIP of 
order K if there exists a constant #(3>) such that 

(l-«(*))||x||l<||#x||l<(l + 5(#))||x||| (6) 

for any A'-sparse vector x. In particular, the minimum of all 
constants <5(<I>) satisfying (|6]l is called the isometric constant 
5k{&)- Also, as a related quantity, the (AT, AT') -restricted 
orthogonality constant Ok,K'(&) is defined as the minimum 
value satisfying 



(*x,*x')| <0x,x'(*)||x|| 2 ||x' 



(7) 
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for all if and if' -sparse vectors x and x' having disjoint 
supports and K + K' < n. In the sequel, we use \i, 
5k, and 9k, K' instead of u(«fr), 6k(&), and 8k,K'(^) f° r 
notational simplicity. Davenport and Wakin showed that, under 
5k+i < 3 Jr^ , the OMP algorithm guarantees the perfect 
reconstruction of any if -sparse signal in if iterations |10|. 
Liu and Temlyakov improved the condition to 5k+i < ^= 
Ifl3l . and Wang and Shim HD and Mo and Shen fl3] further 
improved the condition to 5k+i < ■ They showed 

the optimality of the condition by providing an example that 
there exist if -sparse signals which cannot be recovered under 

5k + 1 = Tic- 
While many studies on the OMP algorithm has focused on 

the case where the iteration number is limited to if, there 

has been some works investigating the behavior of the OMP 

algorithm when it performs more than if iterations 1(161 - 

[18 1 or when it chooses more than one indices per iteration 

[ 19 1. Both in theoretical performance guarantees and empirical 

simulations, these approaches provide better results and also 

offers new insights into the algorithm. Livshitz showed that 

8 a Ki-2 = f3K~ 0/I guarantees the exact reconstruction of K- 

sparse signals within [aJf 12 J iterations [16|. The main benefit 

of this result is that the measurement size ensuring the perfect 

recovery is m = 0(K 16 logn), which is clearly less than 

m = 0{K 2 logn) for the OMP with if iterations iflOl, fill. 

04). Unfortunately, since a — 10 5 and /3 ~ 10" 6 , it is 

not easy to enjoy the benefit in practice. Recently, it has 

been shown by Zhang that S^ik < | guarantees the perfect 

recovery of A"-sparse signals within 30if iterations [17 | and 

this result has been improved by Foucart to S22K < with 

12K maximal iterations ifTBl . The moral of the story in these 

works is that the OMP can actually recover if -sparse signals 

in cif (c > 1) iterations when 8q(k) is upper bounded by a 

constant, which in turn implies that the required measurement 

size is m = O (if logn). 

In this paper, we study the perfect recovery condition of 

the OMP algorithm for two distinct scenarios: when the OMP 

performs 1) strictly if iterations and 2) more than if iterations. 

We henceforth refer to these two schemes as OMPa and 

OMPcK, respectively. The main contributions of this paper 

are twofold: 

• In the first part of this paper, we provide the RIP condition 
for the OMPa, expressed in terms of 8k and Ok.k 1 - We 
show that the perfect recovery of any if -sparse signal can 
be ensured if the sensing matrix $ satisfies \[KQk,\ + 
8k < 1. We also show that this condition is optimal in the 
sense that the perfect recovery is violated even with the 
slight relaxation of the proposed condition. Interestingly, 
the proposed result embraces many of recently proposed 
bounds of the OMPa. For example: 

- By combining the well known "8k — fJ>" inequality 
8k < (K - 1) fi 0, ED, EQl and the proposed 
condition, we obtain the MIP based recovery bound 
V < 2ihi (Theorem ESI . 

- Using the proposed condition together with the sim- 
ple relationship between 9k. k 1 and 8k (6k. k 1 < 
8k+k' lEl), we obtain the RIP based recovery bound 



8 K +i < v =-^ El (Theorem US. 
- When combined with the square root lifting in- 
equality 9 u k.K' < \[u9k,k' 0, we achieve the 
RIP based recovery bound 8 K < ,ir K 7} „ 12TI 
(Theorem |2^1 ). 
• As mentioned in ifTTl . 1181 . the OMP c a has less stringent 
and hence better recovery condition compared to the 
condition of OMPa- In the second part of this paper, we 
establish a new recovery condition for the OMP c a. To be 
specific, we show that if the sensing matrix $ satisfies 
the RIP with ^L 8 .93icj < 0.03248, then it is possible to 
recover any if -sparse signal within 6if iterations using 
the OMP c a algorithm. 
The rest of this paper is organized as follows: In Section 
HIl we present an optimal RIP condition for the OMPa to 
achieve the perfect recovery of if -sparse signals. In Section 
Hill we provide the perfect recovery condition for the OMP c a- 
We conclude our paper in Section [IV] 

We summarize notations that will be used in this paper. 
T = supp(x) — {i\xi 0} is the set of non-zero positions 
in x. O = {1,2,- •• ,n}. For S C. CI, \S\ is the cardinality 
of S. T\S is the set of all elements contained in T but not 
in S. #s € Ji mx \ s \ i s a submatrix of $ that only contains 
columns indexed by S. is the transpose of the matrix $5. 
xg € Ti' s ' is an vector which equals x for elements indexed 
by S. If $5 is full column rank, then = ($' s $5) _1 $' s 
is the pseudoinverse of $5. span($s) is the span of columns 
in $5. Pg = is the projection onto spanks)- P5 = 

I — Ps is the projection onto the orthogonal complement of 
span(&s)- 

II. Perfect Reconstruction of if -sparse signals 
via OMPa 

A. Sufficient Condition 

The main result for the OMPa is given in the following 
theorem. 

Theorem 2.1: Let x G TZ n be any if -sparse vector with 
support T and let $ e R mx " be the sensing matrix satisfying 

VkOka+Sk < 1, (8) 

then the OMPa perfectly recovers x from y = $x, 

Before we proceed, we provide lemmas useful in our 
analysis. 

Lemma 2.2 (Monotonicity property If a sensing ma- 
trix $ satisfies the RIP of both orders if and if', then 
8k < 8k> for any if < if'. 

Lemma 2.3 (A direct consequence of RIP H22V ): Let I C 
CI and be the restriction of the columns of $ indexed 
by I. If S\i\ < 1, then for any vector x 6 T?.' 7 ', 

(l-^i/^llxlla < ||*i*/x|| a < (l + * m )||x|| 2 . 

Lemma 2.4 (Square root lifting inequality For any 

cj > 1 and positive integers if, if' such that ljK ' is an 
integer, 6 uK ,K' < V^0k,K'- 
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Lemma 2.5: Let ii,/ 2 € Q be two disjoint sets (ii n Z 2 = 
0). Then for any vector x supported on Z 2 , 

11*^*^112 <^| Zl |,| 72 |||x|| 2 . (9) 

Proof: See Appendix lAl ■ 

Proof of Theorem O 

Proof: We will prove the theorem using mathematical 
induction. In the first iteration (fc = 1) of the OMPa, an index 
t 1 of the column maximally correlated with the measurement 
y is chosen. That is, t 1 — argmax^ \ (<pi,y}\. Then, we have 



K^,y)l - 
> 

> 



ll*'y||oo 
ll*Ty||oo 

viv 



(10) 

(11) 

(12) 



where (TT2b is due to \T\ = K. Further, since y = $tXt, we 
have 



l(^,y)l > 



— ||*^* T x T || 2 
V K 



> 



-L(l-^)||x T || 2 , 



(13) 



where ([T3l follows from Lemma 12.31 

Now, suppose that t 1 does not belong to the support of x. 
Then t 1 n T — and hence from Lemma [231 we have 



\{<t>ti,y)\ = II^i*tx t | 



< 



(14) 



(1 - 5 K ) ||x T || 2 > 0i,k\\*t\ 



T 



hM\^T\\ 2 - 

This case, however, will never be true if 
1 

\[K 
Equivalently, 

VKdi, K + S K <l- 

In summary, if \J~KB\,k + 5k < L then t 1 <E T for the first 
iteration of the OMPa. 

Now we consider the general (non-initial) iteration of the 
OMPa'- In this step, we show that under the hypothesis that 
the former fc iterations are successful, the (fc + l)-th iteration 
should also be successful. To be specific, we show that if 
T k = {t 1 ^ 2 , ■ ■ ■ ,t k } C T, then t k+1 is in T but not in T k 
(t k+1 e T\T k ). 

First, recall that the residual of the fc-th iteration is given 

by 

r fe =y-*x fe . (15) 

Since supp(x k ) =T l cT by the hypothesis and supp(x) = 
T, one can easily deduce that r k is a linear combination of 
the \T\ (= K) columns of <I?t- Accordingly, there exists a 
i^-sparse signal A = x — x fc supported on T such that 

r fe = * T A T = *A. (16) 

Clearly, r k is the measurements of the if -sparse signal A 
using the sensing matrix Since r k preserves the sparsity 
level of the original signal x, if \[~K9\^k +&k < L the index 
t k+1 chosen in the (fc + l)-th iteration belongs to the support 
of A (i.e., t k+1 £ T). Recalling that the already selected 
index will not be re-selected (see Eq. ©), t k+1 e T\T k . 
This concludes the proof. ■ 



B. Optimality 

Theorem l2. 1 [ characterizes the "success" of the OMPa using 
the isometric constant 5k and the restricted orthogonality 
constant 9k, K'- These two quantities are related respectively 
to the mutual coherence parameter ji by [6|, 0~2), ll20l 



S K <{K-1)h 



and 



9 K ,K> < VKK'fi. 



(17) 



(18) 



By a simple connection between these inequalities and 
Theorem 12. 11 one can obtain the Tropp's MIP condition lTT2l . 

Theorem 2.6 (Alternative proof of \i < 2 k-i Let 
$ be the sensing matrix with unit £ 2 -norm columns. Then 
under // < 2 k-i ' t ^ le O^Pa perfectly recovers any if-sparse 
signal x from the measurements y = 4>x. 

Proof: Plugging Sk < (K — l)fi and 6k,i < VKfJ- into 
VK9k 1 + 5k < 1, we have 



or 



Kft+(K-l)n< 1, 



H < 



2K 



(19) 



(20) 



The optimality of the proposed recovery condition in ([8]) 
is justified by showing that the perfect recovery is violated 
even with the slight relaxation of ((H). To be specific, due to 
the use of inequalities ( fT7T > and ( fT8l in the proof of Theorem 
12.61 the proposed condition is essentially guaranteed by (|20| ) 
and hence better than (less stringent) or at least equivalent to 
(l20llFl Indeed, considering the optimality of condition (TZOdFl 
we claim that the proposed condition is optimal. 

C. Relations to the previous RIP bounds 

The proposed condition consists of two quantities: the 
isometric constant 5k and the restricted orthogonality constant 
9k,k'- In order to obtain the condition expressed in terms of 
the isometric constant only, we use (2) 



'K.K' 



< 5 



K+K' 



(21) 



Combining (I2TI 1 and ((8), we obtain the RIP based recovery 
condition Ifl2l . 

Theorem 2.7 (Alternative proof of 5k +1 < y-^ 1 4741/ ): 
The OMPa recovers any if -sparse signal x from the 
measurement #x if the sensing matrix 4> satisfies 

5 K +i < -= ■ (22) 



(23) 



/K + l 

Proof: Applying K ' = 1 in ((21), we have 

@K,i < 5k+i- 



1 When the bound becomes less stringent, the set of feasible sensing matrices 
# for which the perfect recovery is possible gets larger. 

2 The optimality of condition j2Q\ has been proven in [23 Theorem 3.1] 
by showing that the OMPjf can fail to recover A'-sparse signals under fj, = 
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TABLE I 

Recovery bounds of the OMP k algorithm 



I TspH nr<mpi"tv 

V * V. LI 171 IV 


Rppnvprv linimrl 

1VCLU TCI J UUU1IU 


1? 1'iiiM rk v 

IVLlIliil I\ 9 


MIP 




optimal 




VK6 Kjl + S K < 1 


optimal 


RIP 


s * < JkWIk ED 


near optimal 




s *+i < El 


near optimal 



Using the monotonicity property of the isometry constant 
(Lemma |2. 2k we have 

6k < 6 K +i. (24) 
The theorem is established by plugging d23l and ( l24T i into ([8j. 



Note that the recovery condition in Theorem 12.71 can be 
obtained by imposing two relaxations in (l23l and d24i > on The- 
orem [2T| For sure, when a different relaxation is applied, one 
can obtain the different recovery condition. Indeed, with the 
help of the square root lifting inequality 6 m k K> < V^Gk,K' 
[0, one can obtain the RIP condition expressed in terms of 6k, 
which is presumably smallest order of the isometry constant 
and hence most natural and easy to interpret. 



Theorem 2.8 (Alternative proof of 6k < 



VK^l+K 

The OMPk recovers any if -sparse (if > 1) signal x from 
the measurements y = €>x if the sensing matrix <I> satisfies 



6k < 



1 



(25) 



Proof: Let ui = j^zr, then clearly lj > 1. Using the 
square root lifting inequality in Lemma 12.41 we have 

(26) 
(27) 
(28) 



K,l = 


fui(Jf-l),l 


< 


VuOk-i,i 


< 


\/uj6k 















K, 



(29) 



where d28T > is because 8k,K' < 6k+k'- Using dHJ and j29l , 
one can easily show that d25l l is the sufficient condition for 
the perfect recovery of the if -sparse signal x. ■ 

In Table I, we summarize the recovery bounds of the OMPk 
algorithm. Note that among four bounds in the table, the 
proposed bound is the weakest. 

III. Perfect Reconstruction of if -sparse signals 

VIA OMP cK 

In this section, we analyze the RIP based condition under 
which OMP c k can perfectly recover any if -sparse signal. 
The OMP C K can recover if-sparse signals accurately when 
all indices in the support are chosen within cK iterations. 
Therefore, when the iteration loop of the OMP c ^ is finished, 
the finally selected index set (where / denotes the final 
iteration number) may contain indices not in T. Even in this 



situation, the final result is unaffected and the original signal 
x is perfectly recovered as long as T C (/ < m) becaus^l 



arg mm ||y 



*x|| 



(30) 



and 



(x>) 



Tf = 



x T /, (3D 



where PIT ) follows from the fact that x T /^ T = 0. 

As mentioned, it has been shown in fTTll . |[T8l that T C 
T 30K and T C T 12K under the condition 6 31K < | 
and 622K < g, respectively@ Unfortunately, these conditions 
impose a bit too many iterations and also cause a limitation 
on the sparsity level K. For example, 12if iterations require 
the sensing matrix to be at least 12if rows (m > 12K), which 
in turn requires K < m/12. In short, our main result is that 



T C T 6K holds true under 8 



[8.93KJ 



< 0.03248. Roughly 



speaking, our result indicates that the OMP c ^ can perform 
accurate reconstruction of if -sparse signals with only half 
number of iterations of what has been proved thus far. Our 
main result is formally described in the following theorem. 

Theorem 3.1: Let x G 1Z n be any if -sparse signal sup- 
ported on T and $ G TZ mxn be the sensing matrix with 
unit ^2-norm columns. Further, let T k — T\T k be the set 
of remaining support indices after k iterations of the OMP c ^. 
If ^[8.93^] < 0.03248, then T C T fc + 6 l rfe l. 

The key point of the theorem is that after k iterations 
of the OMP C K, the remaining support indices in T k will 
be selected within 6|r fe | additional iterations as long as 
8 l8 . 93K i < 0.03248. 

Corollary 3.2: Let $ £ TZ mxn be the sensing matrix 
having unit ^-norm columns and let x £ lZ n be any K- 
sparse vector with support T. Then, under <5[8.93i<"J < 0.03248, 
the OMPcK perfectly recovers x from y = 3>x within 6if 
iterations. 

Proof: Applying k = in Theorem[34] T k = T\T° = T 
and hence T fc+6 l rfc l = T 6 I T I = T 6K DT. ■ 



A. Preliminaries 

Before presenting the proof of Theorem 13.11 we provide 
some useful definitions, lemmas, and propositions. Without 
loss of generality, we assume thatr fc = {1,2,--- ,|r fc |}.Then 
it is clear that 1 < |r fe | < if. For example, if k = 0, then 
T k = and |r fc | = \T\ = if. Whereas if T k D T, then 
r fc = and |r fc | = 0. For notational convenience, we assume 

3 The condition / < m is required for solving the LS problem (3D) . 

4 Note that there are two errors in Proposition 5 of [18. p. 5], where the isom- 
etry constant Sry^r^i p 2-\\j^ and the iteration number n = 2[3/p 2 ] |T\T°| 
should be replaced with 5(i+7[3/ p 2-|) X and n = 4[3/p 2 ] |T\T°|, respec- 
tively. As a result, the condition for the OMP c k (i.e., when p = 1) become 
&22K < 1/6 with 12K maximal iterations \24\. 
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that {xi}i = i 2 ... ,|r fc | are arranged in descending order of their 
magnitudes, i.e., 

\xi\ > \x 2 \ > ■■■ > \x\ r *\\. (32) 
Also, we define the subset T k of T k as 
f0 r = 0, 

rj= |{1,2,- - .2- 1 } r = l,2,... ,[log 2 |r fc n, (33) 

See Fig. 1 for the illustration of T k . Notice that the last set 
^T[iog | r fcp +1 may have less than 2^ log2 ^ r " elements. For 
example, if T k = {1,2,- ■■ ,7}, then = 0, Tf = {1}, 
r* = {1, 2}, T k = {1, 2, 3, 4}, and T k = {1, 2, • • • , 7} = T k . 

Lemma 3.3: For a given set r fc and a constant a > 2, let 
ifc,o- € {1, 2, • • • , |~log 2 |r fc |] + 1} be the minimum positive 
integer satisfying^ 



I x r & \rgll2 

2 
2 



r fc \r*M 



< 
< 



°1l x r*\rtll2) 

2 
2' 



(7\ \ Xpfc 



r fc \r£ 



(34) 
(35) 



n 






7* (support set) 










(estimated support set) 





(a) Set diagram of T, T k , and F k 




flog, |r* |] + 



r r 

(b) Illustration of indices in T k . 
Fig. 1. illustration of sets T, T k , and V k . 



< 
> 



cr M x r fc \r£_ 1 II2? 



r fc \rj 



(36) 
(37) 



Then, 



1) ir*| < |r fc i 



,L-1 



iL-2 



2) |r||<2 i 

3) |r fc | > 2 L ' 2 , 

4) |r fe l>(§f^)2' 

Proof: See Appendix IE1 ■ 

The following proposition provides a lower bound of the 
residual power difference ( 1 1 r z 1 1 1 — ||r' +1 || 2 ) in the (I + l)-th 
(I > k) iteration of OMP cK - 

Proposition 3.4: Let x E 1Z" be any i-T-sparse vector 
supported on T and <fr g j^mxn ^ e ^ sensm g matrix with 
unit £ 2 -norm columns. Then, for a given T k and an integer 
I > k, the residual of the OMP c k satisfies 

i-v^, (||ri||2 _ ||$rfc ^ xrfe ^ ||2) 

(38) 



„/ 11 2 



^+l||2 



> 



|r*| 



where r = 0,1,- •• ,riog 2 |r fc H +1. 
Proof: See Appendix ICl 



From Proposition 13.41 we further obtain the following 
proposition, which plays an important role in the proof of 
Theorem 13.11 



(39) 



Proposition 3.5: For any integer I' > I and r g 
{0, 1, • • • , |~log 2 |r fc |] +1}, the residual of the OMP cK satisfies 

||^||2<C , r,i,i'll> J ll2 + (l + V*|)ll x r*\r*lla, 

where 



C r iy = exp 



(i'-0(i-*i 



|r* ur i'-i| 



|r*| 



(40) 



Proof: See Appendix ITJ1 



B. Proof of Theorem 13.71 

Our proof is based on the mathematical induction of the 
number of remaining indices |r fe | (after k iterations). We first 
check the case when |r fc | = 0. This case is trivial since it 
implies that all support indices have been selected (i.e., T C 
T k ) so that no more iteration is required. Next, we assume 
that the argument holds up to |r fe | = 7 — 1 (1 < 7 < K). In 
other words, we assume that if |r fc | = j (1 < j < 7 - 1), 
then maximally 6j additional iterations are required to select 
all the support indices in T k . Under this assumption, we show 
that if |r fe | = 7, it requires maximally 6|r fe | = 67 additional 
iterations to find out the rest of support indices (i.e., f—k < 67 
where / is the final iteration number such that T C T*). 

Although the details of this proof are somewhat cumber- 
some, the main idea is simple. When |r fe | = 7, we first show 
that the decent amount of indices in T k can be selected within 
a specified number of additional iterations. To be specific, let 
h = k+ [1.5(1 + EUil r r 1)1 fori = 0,l,--- ,L(T* andL 
are defined in (l33l and Lemma l3~3l . Then, we show that under 
$\TuT k L\ < 0.03248, the OMP c k algorithm selects more than 
2 L ~ 2 support indices in T k within k^ — k additional iterations 
(i.e., from the (fc + l)-th to the fc^-th iteration). In other word, 
the number of remaining support indices in T kh (after kz 
iterations of the OMP c ^-) satisfies 



\T kL \ < 7 -2 



L-2 



(41) 



Second, we show that by the induction hypothesis, it requires 
no more than 6|r fel -| iterations to select the rest of support 
indices in T kL (i.e., f - k L < 6\T kL \). In summary, the goal 
of the induction step is to show 



In the sequel, we use L instead of Lfe j(T for notation convenience. 



k L + 6(7 - 2 W ) - k< 67. 



(42) 
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Number of remaining support indices 




First, we find out an upper bound of ||r fcL |||. Since |r fc | = 7, 
by applying Proposition 13.51 we have 



" k <6 / f lt era tion number 

(a) In the induction step, we show that when |T fc | = 7, / — k < 67. 
Number of remaining support indices 




6|rM 



f Iteration number 



\v k -\ < |r*\r*_ x | 



(43) 



Further, recalling that {aft}i=i,2,— ,7 are arranged in descend- 
ing order of their magnitudes (i.e., \x%\ > \x%\ > ■ ■ ■ > |x 7 |), 



it is clear that x 



consists of 7 — 2 L 2 most non 



significant elements in xpt . Hence, one can easily check that 
the sufficient condition of d4lT> is 



M^t II 2 



< x 



(44) 



To show this, we construct upper and lower bounds of \\r kL ||§, 
as 



-kill 2 



\r kL \\ 2 2 > BeU 



X r *,\ r fc_ i 

2 

r k L ||2- 



2 

\2i 



(45) 
(46) 

Since d44l holds true under B u < Bg and also (l44l > is 
a sufficient condition of (t4TT >. it is clear that (l4lT > can be 
guaranteed as long as B u < Bg. 



\\r kl \\l < Ci tkoM \\i*\\l + (1 + ^)11x^11^ 



fe||2 



(47) 



Jfe II 2 



< 



C2 )fcl ,te||r tel |l2 + (l + ^)l|xr fc \rjll2: (48) 



llr^H 2 , < C L , kL _ uk Jr k ^\\l + (l + 5,)\\^ rkxr ,jl 

(49) 

Noting that h = k+ \lMl+J2l =1 |r*|)l for i = 0, 1, • • • , L, 
it follows from (0O]l thaO 



Ci.fe 



exp 1 



exp i 



exp 1 



1.5(1 + |r;i)(i-<s |r?ur » 1 - 1| ), 

|I*| 

1.51^1(1-^^-1^ 

|r?| 

i.5nrj|l(i- y £ur ^-i|) 
|r|| 



, i = 1, 



» = 2,..- ,L-1, 



), i = L. 



(50) 



One can easily show that for i = 1, • • ■ , L, 

1.51^1(1-^^-1^ 



Cifa^M < exp(- 



(51) 
(52) 



(b) Two ingredients in the proof of induction step: 1)7 — \T kL \ > 2 L 2 and 

2) f-k L < 6|r fe H 

Fig. 2. Illustration of the induction step. 



An illustration of the induction step is described in Fig. 2. 

Before we proceed, we briefly explain the key steps to prove 
(|4TT >. Consider x r k L and x rfc \ r fc , which are two truncated 
vectors of x r f=. From the definition of T k in ( f33T >, we have 

r*_j = {1, 2, • • • , 2 L ~ 2 } and r fc \r*_! = {2 L - 2 +i, • ■ ■ , 7} 

so that an alternative form of d4lT i is 



< exp (-1.5(1 -5| r j UT k 4 -i|)) 

< exp (—1.5(1 — £|r*uT*x.- l |)) (53) 



where d53l l is from the monotonicity of isometry constant in 
Lemma I2.2p| 

For notational simplicity, we let 77 = 

exp (-1.5(1 - <J|r*uT*t- 1 |))- Then EMU) can be 
rewritten as 



I|r fcl |l2 < »?||i*||| + (l + MI*r*\r*ll! 
||r^|| 2 < ^||a + (l + * 7 )|| Xrni *|] 



2- 



(54) 
(55) 



< ,||r*^|^ + (l + 5 7 )||x r » xr£ ||^ (56) 

Multiplying ( f54b by ■q L ^ 1 and ( f55l l by ?/ L ~ 2 , and so on, and 
then summing these rows, we have 



|r^|| 2 <^||r fc || 2 + (l + <5 7 ) 



L 

\ " L- 



1! £^ V 

r=l 



|x r *\r*ll2- (57) 



In order to obtain an upper bound of \\r kL |||, we need to find 
out an upper bound of 1 1 1-^ 1 1 § - 



6 Since \F k \ = 2 i ~ 1 for i = 1, 2, ■ ■ ■ , L - 1, we have fci - &o = 
[1.5(1 + |rf|)l = 1.5(1 + \T\\) and fc» - = k + [1.5(1 + 

eu ir*Di - (*+ ri.5(i+E;=\ ii*i)u = ri.5(i+Eu ^-^i - 
ri-5(i+Er=i 2 * _1 )i = r!-5-2 i i- ri.5-2 1 - 1 ] = ls^-ls^*- 1 = 

1.5 ■ 2*" 1 = 1.5|rf I for i = 2, • •■ , L - 1. 

7 Since r k C rf and T ki ~ 1 C T* £_1 , we have rf U T*^ 1 CTjU 
T*^- 1 and |rf UT'-^I < [r* UT*'" 1 !. 
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Lemma 3.6: The residual r satisfies From the monotonicity of isometry constant, we have 

\\r k \\ 2 2 < (1 + M|x rnrg |||. (58) |r fc |<|TUT fei | <5 7 < 5 |TuTfcl |, (74) 

Proof: See Appendix |E] ■ l r l u T kL ^\ < \T U T kL \ ViuT^- 1 ! < &\TuT k L\- 

Plugging gill into (g7j yields ^ 75) 

l From d74l d75l) . and d73T >. we have 

||r fc -||2<(l + <5 7 )5>M|x rfcxr ,||2. (59) 

Also, from the definition of L in Lemma [331 we have 1 — &\TuT k L\ 

2 L-l-r 2 (?6) 

||x rJ! \ r fc|| 2 < er 1 T ||x rfcvr fc_ i ||^ (60) When S ]TuT k L] < 0.03248, one can easily show that 



4 (1 + s \tut-l\) exp (-1.5(1 - 5| TUT *z,|^ 



for r = 1, 2, • • • , L. Applying (|60]l to (|59]l yields 



4(1 + <5| TuT fc L i ) exp (-1.5(1 - S lTUT k Ll )) 
(l + 5 7 )||x rnr , (— ^<1, (77) 

< x y^M) (6i) i-«itut^i 



|r^||| 

T=0 



which implies B u < Bi and hence we can conclude that (HTt 

|TuT fc i| 

We are now ready to prove the induction step. As shown 

When < a V < 1, we further have in GD» the number of remaining support indices after k L 

iterations satisfies | T kL \ < 7- 2 L . Since 7- 2 L <7~ 1, 



(1 + <5 7 ) ||x r h\ r fc_J|| ^ ^ holds true under <5| TuT ^| < 0.03248 



a 

T=0 



(1 + S y ) ||x r fc^ r fc J| 2 °° we also have 



r fcz -| < t - 1- (78) 



fc < - — U ^ E(^) T c«) 

(T * — ' 

T = 

(1 + <5 7 ) 2 Xhe induction hypothesis implies that all indices in T kL can 

= — T- r x r fc \r fc k>i (.04) , ; , ra 

cr(l - or)) Xi t-i be chosen within 6|r fci | additional iterations of the OMP cA |!l 

which is the desired upper bound of 1 1 r fex - 1 1 2 (see d45]l). That a * * S ' , 

is> " " 2 /-fcL<6|rH (79) 

B u = t — ^ 7 ■ (65) Recall that our goal of the induction step is to show that it 



<r(l — arj) 



requires 67 maximal iterations to select all the support indices 



Second, we build a lower bound of ||r fe - L || 2 ! . Noting that in T , i.e., 

|x-x fei || < \TUT kL \, we have " " / - k < 67. (80) 



c kL \\l = ||*(x - x kL )\\l (66) From |79]l, it is clear that ([80]i holds true whenever 

> (1- 6 lTUTkL{ )\\x-x kL \\l (67) jfei + 6|r fci | -fc<67. (81) 

> (l-5|Tur*t|)ll x r*t||i- ( 68 ) TT . __ 

Using ( 1411 ). we have 



Thus, 

B t = l- S {TuTkL , . (69) k L + 6|r fc - 1 - k < k L + 6(7 - 2 L - 2 ) - k. (82) 

As mentioned, it suffices to show B u < B e to prove gl). Now > if we show that 
Using m and ©, we have ^ + g(7 _ 2 l- 2) _ fc < ^ (g3) 

■B« = — 7-: rrz — r Bi. (70) then ( f80b holds true and the induction step is established. 



cr(l - <7T])(1 - 5 lTuT k Ll ) 

Note that < err] < 1 and 



From the definition of k^, 



1 / a 2 1 ^ = fc+ri-5(i+Ei r ri)i (84) 

<7(l- < 77 7 ) = — Uj-- +-. (71) f^i 



V V 2 / 477 



L-l 



Hence, by choosing (777 = 1, <r(l — arj) takes the maximum = k + [1.5(1 + |r*|) + 1.5|r* |] (85) 

value ir- and r=i 



4i; 



L-l 



Bu = _tt±*rL ^ (72) = fe+[1.5(l + ^2- 1 ) + 1.5|r||l (86) 

1 — 0\TuT k L I r = 1 

4(l + & y )exp(-1.5(l- Vut^-m)) = k + ■ 2 L ^ + 1.5\T k \] , (87) 

= — ; — " Be ~ 

\T{-1T l\ 8j n ^ ; n( j uc ti on step^ we assume that when |r fe | < 7 — 1, it requires 

(73) 6|r fc | maximal iterations to find out all the indices in T k . 



s 



where (|86} uses the fact that |r£| = 2 T_1 for r = 
1, 2, • • • , L — 1. Then, it follows that 

fc L +6( 7 -2 L - 2 )-fc 

= [1.5 • 2 1 - 1 + 1.5|r£|] +6(7- 2 L " 2 ) (88) 

< [3 - 2^-— 1_ | +6( 7 -2 L - 2 ) (89) 

= 3-2 L " 1 + 6( 7 -2 i " 2 ) (90) 

= 67, (91) 

where (l89l is from |T| | < 2 i_1 in Lemma [3~3l This completes 
the induction. 

In summary, we have 







(92) 


D 


yfc+(fci,+6(7-2 L - 2 )-fe) 


(93) 


D 


27fc+(fcx.+6|r fc i|-fc) 


(94) 




yfci,+6|r' t i| 


(95) 


D 


T f 


(96) 


D 


T, 


(97) 



where (|93]l is from (J9TJ, (O is due to d82]i, and (|96]l is from 

Thus far, we have shown that under S\tut<*l\ < 0.03248, 
T C r fc+6 l rt: l. We now need to find an order \T U T kL \ of 
the isometry constant. In this step, we first show that |T U 
T fei I < fc+ [3.97] and then show that k + [3.9 7 ] < [8.93if|. 
Combining these two inequalities, we obtain |T U T kL \ < 
L8.93KJ. 

First, observe that 



\TUT kL \ = \T\T kL \ + \T kL \ 

= |r fci | + |T fci | 

< !-2 L - 2 + k L , 



(98) 
(99) 
(100) 



where (1100b follows from (|4TJ. Combining ([87j and ( 1100b 
yields 

|TUT fe£ | 

< 7 -2 L - 2 + fc+ [l.5-2 i - 1 + 1.5|rf,|] (101) 
= 7 - 2 L " 2 + k + [2 i ^ 1 + 2 L ~ 2 + 1.5|r||] (102) 
= 7 + 2 i_1 + k+ [l.5|r||] (103) 

< fc + 2 L - x + [2.5 7 1 , (104) 

where J 103b is because L > 1 and hence 1.5 • 2 L_1 is an 
integer and dl04b is due to |T|| < |T fc | = 7 in Lemma |331 

Recalling that arj = ^ and rj = cxp(— 1.5(1 — 
^|r£uT fe t- 1 |))' 



a = 0.5 cxp fl.5(l — <5|r*uT fc i- 1 |)) 
< 0.5e 15 

= 2.2408. (105) 



Using this together with property 4) in Lemma 13.31 we have 

(2a - 2 s 



>L-2 



< 



\2o- 1 
(2a- 2 
\2a-l 

7 

7- 



7 



2a -I 
< 0.71287 



and 



2 L - Y < [1.4256-yJ. 
Finally, using dl04b and ( 11071 ), 

\TUT kL \ < k + 2 1 - 1 + [2. 57] 

< k + Ll.4256 7 J + [2- 5 7l 

< k + [3.92567] 

< k+ [3.937] (108) 

Next, we build an upper bound for k + [3.937]. As 
mentioned, k is the number of iterations already performed 
and |r fc | = 7 is the number of remaining correct indices. In 
order to find an upper bound for k + [3.93 7 ], the following 
three cases need to be considered. 

• k = (i.e., in the initial iteration): In this case, it is clear 
that 7 = K and hence 

k + [3.937] = 3.93X. (109) 

. < k < 5K: Since 7 < K, 

k + [3.937] <5K+ [3. 93.fi:] = [8. 93.fi:] . (110) 

It is worth mentioning that the case of "k — 5K, 7 = K" 
corresponds to the worst case where the OMP c jf does not 
select any support index for the first 5K iterations. Since 
the required number of iterations cannot exceed 6K, the 
remaining K support indices should be selected in the 
last K iterations. 

• 5K < k < 6K: Again, since the required number of 
iterations for selecting K support indices cannot exceed 
6K, we must have 7 < 6K — k and thus 

k+ [3.937] < k+\3.93(6K-k)] = [23.58if-2.93A:]. 

(HI) 

Since k > 5K, we further have 
k+ [3.93 7 ] < [23.58.fi:- 2.93 -5.fi:] = [8. 93.fi:]. (112) 
In summary, for all cases 

k+ [3.937] < [8.93X]. (113) 
Using this together with ( 11081 ), we have 

\TUT kL \ < [8.93.fi:j, (114) 
which completes the proof of Theorem 13.11 



(106) 



(107) 
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IV. Conclusion 

In this paper, we investigated the recovery condition of the 
OMP algorithm ensuring the perfect recovery of sparse signals 
for two distinct scenarios. In the first part of this paper, we 
have provided a simple bound of the OMP guaranteeing the 
exact recovery of A-sparse signals using A"-iterations of the 
OMP algorithm. Our condition, \K6k,x + $K < 1, bridges 
the gap between MIP and RIP conditions and also embraces 
many of recently proposed optimal or near optimal bounds of 
the OMP algorithm fl2l. |[T4t |2fl. 

In the second part of this paper, we have analyzed a recovery 
condition for the OMP algorithm when the iteration number 
is allowed to be more than A. We have shown that the 
OMP can perfectly recover any A"-sparse signal within 6A 
iterations under the RIP of order [8.93A"J, which improves 
the previous results (30A iterations in [17 | and 12K iterations 
in lfl8l ). Considering that larger number of iterations results in 
higher computational complexity and also imposes on stricter 
limitation to the sparsity level A, the reduction on the iteration 
number offers practical benefits in complexity as well as the 
relaxation of the sparsity of underlying signals to be recovered. 

Appendix A 
Proof of Lemma |231 

Proof: Let u g w 7 '' be a unit ^ 2 -norm vector, then 
max |(* /lU ,* /2 x /2 )| 

u: u 2 = 1 



max ||u'(*^*7 2 x/ 2 )||2 

u:[|u||a=l 

= ||*' 7i * j2 x l2 || 2 , 



(115) 



where the maximum of the inner product is achieved when u 
is in the same direction of <&j <&j 2 xj a , i.e., 



||*k*/ a x /a || 2 - 

Moreover, by the definition of restricted orthogonality con- 
stant, 

|(* /l u,* /2 x /2 )| < e»| /lMJ2 |||u|| 2 ||x|| 2 

= #| /lM/2 |||x|| 2 . (116) 
We obtain the desired result by combining ( II 15b and (11161 l. 



Appendix B 
Proof of Lemma [3731 

Proof: The first three inequalities are trivial. The last 
inequality can be justified as follows. From d34l ). we have 



\l<((T-l)\\^\rlJ\l (H7) 



Recalling from the definition of r£ and L (see (l33l and 
Lemma I3.31 l. 



r|_!\r|_ 2 = {2 



L-3 



r fc \r*_! 



{2 



h-i 



i,---,|r fc |}. 



(118) 
(119) 



Noting that {xi} i= i 2.... ,|r fc | are arranged in descending order 
of their magnitudes, the elements of x Tk \ r k are |r fe | — 2 L ~ 2 
most non-significant ones in x r t (and hence smaller than the 
elements in x r t * r * ). Since a — 1 > 1, we obtain from 
dTT7l that 



n- 1 \n_ 2 \<(a--i)\r k \r k L _ 1 



That is, 



2 L - 3 < (cr — l)(|r fc | — 2^-2), 

which completes the proof. 

Appendix C 
Proof of Propositions. 41 



(120) 
(121) 



The following lemma is useful in proving Proposition [ 

Lemma C.l: Let Ar = r' — r l+1 . Then the OMP c ^ satisfies 
the following equalities. 

1) Ar = P T i+ir l , 

2) Ar = P^*x'+\ 

3) Ar = P ^+ 1 x^ 1 1 , 

4) ||Ar|| 2 = ^<r' 1 tl+ ix[+ 1 1 ). 
Proof: 

1) Since r' +1 = y — P T i+iy = P™ +1 y, we have 



Ar = r' - P x 



= r , -P£ +1 (r , +*x') 



(122) 
(123) 

= r'-P^+V (124) 
= P T;+ ir z (125) 

where ( 11231 ) follows from r' = y — <I>x', (11241 ) uses the 
facts that <frx' = Pj^y lies in span(^ T i+i) and that 
Pyi+i cancels all components in span(& T i+i). 

2) From r' +1 = y — $x' +1 , we have 

P^*x' +1 = P^y-P^r' +1 



r i _ r /+i 



(126) 

P T ,r i+1 (127) 
= Ar, (128) 

where ( 1128b is because r' +1 is orthogonal to span($> T i ) 
(i.e., P T! r /+1 = 0). 

3) Noting that <&x' +1 = t&^x^ 1 + ^i+ix^, we have 

(129) 

P£ l (* T ix£ 1 + &,+ixJ+ 1 1 ) (130) 

(131) 



Ar = P^*x /+1 



(132) 



where ( 1131b is from the fact that P^, cancels all 
components in span($> T i) (i.e., P^,$ T i = 0). 

4) Since ifitf+^J+i lies in span(4> t i+i), we have 
Pi ! +i0t ! + lX ji+i = <^t ! + lX t i+i 

(r'.&.+ixj+i) = (r , ) P t , + i^, + ixi+ + 1 1 ) (133) 
= (P ii+1 r',0 tI+ ix^+i> (134) 
= (Ar.^w-ixJ+i), (135) 



and 
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n 



Tr^T k u rj (support of w) 




(estimated support set) 



(support set) 



Fig. 3. Illustration of sets T, T , and supp(w). 



where (1134b is because P 4 fc+i = (P t i+i)' and (1135b is 
due to ( 1125b . Using I = P^, + P T i , we further have 



(Ar ) P^ fI+ ixJ+ 1 1 ) + (Ar,P T! ^ t(+ ix^+i) 



(Ar, Ar) 
l|Ar|||, 



(136) 
(137) 



where ( 1136b uses the fact that the vector Ar is orthogo- 
nal to span(& T ,) and hence (Ar, P T i t i+ix^\ ) = 0. 
This completes the proof. 



We are now ready to prove Proposition 13.41 
Proof: From Lemma IC.ll we have 



||Ar|| 



(r'.&.+iitffi) 
l|Ar|| 2 
(r'.^+ixj+j) 

||p^ ( ^ 1+ ixj+i|| 2 



(138) 
(139) 
(140) 



Noting that HPi^i+i || 2 < \\4>t'+ 1 II2 = 1 (3> has unit 
^2-norm columns) and t l+1 corresponds to the maximum 
correlation between r' and the columns in <!?, we have 



||Ar|| 2 ><# I+1 ,r l > = ||*V' 



(141) 



Since r' is orthogonal to span(& T i), <&' Tl r l = and hence 



|#V| 



*'r')n\T' 



(142) 



For a given r fe , let T k C T k where r <E 
{1, 2, • • • , [log 2 |r fc |] +1} be the set defined in <|33}. We define 
a sparse vector w G 7?.™ which equals x for elements indexed 
by TnT fe ur^ (i.e., w TnT fc ur fc = x TnT k ur k) and otherwise. 
We consider the correlation (($'r l ) n \ T i , (w — x l ) n \ T i ). From 
Holder's inequality, 

w x )n\T ! ) 
< ||(*V) nVT ,|U||(w-x') nVT ,||i. (143) 



Since x' is supported on T l , we have ('x 1 )ci\t 1 = ar, d hence 

> 



((*V) n \ T i,(w-x')r2\T'> 



||(w-X%\T'Hl 



((*V) OXT! ,(. 



l )n\T l ) 



ll w n\T ! Hi 
($'r' w — x z ) 



> 



ll w n\T' Hi 
(<f>'r l , w — x z ) 



(144) 
(145) 
(146) 
(147) 



l w O\T'Ho ||w n \ T (||2 

where (1146b is due to (<&V) T i = and (1147b follows 
from the norm inequality ||v||i < \/||v||o 1 1 v 1 1 2 - Noting 
that supp(w) = T n T k U T k and T k C T ( , we have 
supp(w n \ r! ) = (T n T fe U r*)\T' = T k \T l and hence 



||w nXT ,|| = |r£Yr'|<|rJ|. 

Applying (1148b to ( 1147b . 

($'r', w — x') 



S'r'jnvHIoo > 



l r rl ll w n\T'l|2 

Combining (fl4TTl . (fl42l . and (fT49l yields 

($'r', w — x') 



l|Ar|| 2 > 



IT?! l|w n \T'l|2 



(148) 



(149) 



(150) 



Furthermore, we observe that 



(*y,w-x ; ) 



(*fw -x'),r iN 



~(||*(w-x 



iM|2 
2 



J||2 



*(w -x z 



= -(||*(w-x')||^+||r ( ||^-||*(w-x)||^ (151) 



- (||#(w - y>)\\* + ||r'||i - ||* rnr? x rnr ,||^) , 

(152) 



where ( 1151b uses the fact that <&x' + r l = y = <I?x. When 

||r'||| — ||# r fc\ r fcX r fc\ r fc HI > 0, we apply the inequality \{a + 
b) > \fab~ to ( 1152b to get 

(*V,w-x z ) 



> ||*(w-x ! )|| 2 ^llr'Hl-ll^rnr^rjIll. 

(153) 

Noting that w — x' is supported on T k UT l , we have 

||*(w- x')|| 2 

> yjl - VjuT'i Hw-x'Ha (154) 

> ^1 - (5| r fc UT !| ||(w-x%\ T( || 2 (155) 

(156) 



yjl - <5|rfcuT'| ||wn\T'||2, 
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where (1156t is due to (x') n \ T i = 0. Plugging ( 1153t and (|156t 
into ( 1 1501 ), we have 



|Ar|| 2 > 



(1 - V^ut'|) (ll r III - ll*r»\r*xr*\r*lla) 



|r*| 



Further, since HArHl = ||r'||| — ||r i+1 |j2, we have 

||„i||2 ||_i+l||2 



(157) 



1-<J 



> 



|r;uT'| /,, , ||2 



|r*| 



(llr'Hi- ||*r*\r*x r *\r*||i) -(158) 



Next, when \\r l \\l — ||* r k\r*x r *\r* ||| < 0' fl5D holds nat- 
urally since ||r'||| — ||r' +1 ||2 > (the residual is always non- 
increasing due to the orthogonal projection), which establishes 
the proof. ■ 

Appendix D 
Proof of Proposition [33] 

Proof: Subtracting both sides of ( T38l > by ||r'||| — 
||*T\r fcX T\r fc III' we nave 

l|r' +1 ||| - ||*r»\i*x r *\r*ll2 

< (l - 1 ~^ Ur ' 1 ) (ll rZ lll - Il*r»\r*x r *\r ? ll5). 

(159) 



Noting that 



1 



1 — Sir* 



|rfc UT i| 



|r*| 



< exp 



|rfc UT i| 



ir*i 



we have 

J + l||2 



r 1 1 2 - Il*r fc \rfc x r fc \rjll2 



< exp 



1-5 



l r x UT! l \ niJn2 



(II' Ha - ll*r*\r*x r *\r*||S) 
(160) 



and also 

||r ;+2 |||-||* r , xr ,x rfcxr ,||| 
< exp I — 



|r*| 



^+l||2 



i " n*r fc \r* x r fe \r^ II2)' (161) 



ll r ' III - ll*r fc \r^ x r fc \rjlll 



< exp 



1 5\y-k 



| r fc uT ('-l| 



x r 



|r*| 
— 1 1 1 2 



|* rfc \ r fcx rfc \ r fc|||). (162) 



By multiplexing each row, we obtain 



r II2 - II *r fc \r$ x r fc \rj II 2 
l'- 1 / 1 x 

TT / 1-0] 

< [I exp 



|rfc UT i| 



x (Hr'lll - ||* r * Nr *x rnr *||l). (163) 



Since S\ r kuT'\ < d\r$uT>+ 1 \ < •• < ^\r k ur l '- 1 \' we further 
have 



r ' III _ Il*r fe \rfc x r fc \rfc|l2 
< CV,j,i'(||r*||! _ ||* r *\r* x r*\r*ll2). 



where 



C T ,i,i> = exp 



(164) 



(165) 



Noting that CV^;/ > 0, we derive from dl64t that 

||r r ||| < C T , hV \\Al + ||*r*\r*x r *\r*||i. (166) 
Furthermore, since \T k \T k \ < |r fc |, the RIP implies that 

||r ,/ ||l<CV,i,i'll' , lll + (l + Vl)H x r*\r?lll, (167) 
which completes the proof. 



Appendix E 
Proof of Lemma [3~61 

Proof: r'nTC T k implies that 

ll„fc||2 



|P^y||| < HP^nryl 



(168) 



Also, noting that P^; fcnT y is the projection of y onto the 
orthogonal complement of span($> T k nT ), 



|P^ nT y|||= mm ||y - *v||* 

supp(v)—T k nT 



Thus, 



Ir fc ||2 < 



y — <I> T fc nT x T fc nT || 2 
I^txt — *T fc nT x T fc nTl 



= ||* rfc x rfc || 2 
< (l + ^)||x rfc ||| 

= (l + MKnrglll; 



(169) 



(170) 
(171) 
(172) 
(173) 
(174) 



where (TT721 is from T\(T k flT) = T\T fc = T fc , (TT731 is 
from the definition of RIP where |T fc | = 7, and ( I174t is due 



to r§ = 
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